FIG. 1 illustrates in block diagram form an image/video coding system. The data format at each process is shown in FIG. 1. Camera 101 captures an image of the objects within its view. This image is transmitted to down-sampler 102 in the 4:4:4 format. Down-sampler 102 transforms each image into the 4:2:0 format. Image/video encoder 103 encodes the image data into a form for transmission or storage. This typically includes some form of data compression. The resulting bitstream is transmitted and/or stored (transmission/storage 104). Ultimately this bitstream supplies image/video decoder 105. Image/video decoder 105 decodes the image data and recovers the images in the 4:2:0 format. Up-sampler 106 converts this 4:2:0 format image data into 4:4:4 format image data. Thus down-sampler 102 is placed between camera 101 and image/video encoder 103 at transmitter side. Up-sampler 106 is after image/video decoder 105 constructs the image. Note that both 4:4:4 and 4:2:0 represent the image formats of YCbCr color space. Generally, the 4:4:4 format image has more information than the 4:2:0 and thus requires a larger data rate.
FIG. 2 illustrates a comparison of the spatial resolution of the 4:4:4 and 4:2:0 formats. The Y component is called luminance (or luma), while Cb and Cr components are the chrominance (or chroma). Chrominance components Cb and Cr are related to respective blue and red color planes. Each 2-by-2 luminance block 201 in the 4:4:4 format is the same as the corresponding 2-by-2 block 211 in the 4:2:0 format. On down sampling a 2-by-2 block 203 of chrominance component Cb becomes a single block 213 of data. This conversion typically averages the data of the four pixels of 2-by-2 block 203. Alternatively, this conversion could be a filter function of the four pixels with corresponding filter coefficients. This is known as down-sampled with a ratio of 4:1, or more precisely, 2-by-2 to 1. On up sampling, a single block 213 becomes a 2-by-2 block 203. Typically the data of the single block 213 is copied to each of the four blocks of 2-by-2 block 230. Chrominance component Cr has a similar 2-by-2 block 205 in the 4:4:4 format and a corresponding single block 215 in the 4:2:0 format.
FIG. 3 illustrates the position of luminance and chrominance pixels in the 4:4:4 format. In FIG. 3, X represents the position of a luminance pixel and O represents the position of a chrominance pixel. Each 2-by-2 block 301 includes 4 luminance pixels 303 and four each of chrominance Cb and chrominance Cr pixels 305. As shown in FIG. 3 the luminance pixels 303 are coincident with chrominance pixels 305.
FIG. 4 illustrates the position of luminance and chrominance pixels in the 4:2:0 format. Each 2-by-2 block 401 includes 4 luminance pixels 403 and one each of chrominance Cb pixel and chrominance Cr pixel. Luminance pixels 403 are disposed in the same pattern as luminance pixels 303 illustrated in FIG. 3. The two chrominance pixels can be disposed in more than one location. Chrominance pixels 405 (circles) are disposed between the four luminance pixels 403 and centered in the 2-by-2 block 401. This sampling pattern is used in MPEG-1. Alternatively, chrominance pixels 407 (squares) are disposed on the center left of block 401. This sampling pattern is used in MPEG-2. Other possible chrominance pixel locations are illustrated at 409 (triangles).
FIG. 5 illustrates how the chrominance signals change in a conventional system during the image/video coding. The example of FIG. 5 employs an 8-by-8 block. Such an 8-by-8 block is widely used in image/video coding algorithms relevant to this invention. This does not jeopardize a generality of the discussion, one may use any other unit.
FIG. 5 includes line 501 over the original 8-by-8 chrominance block 503. Line 501 marks an edge (i.e., steep transition in gray levels) along the line. The 4:4:4 format data of 8-by-8 chrominance block 503 is down sampled to a 4-by-4 chrominance block 505 in the 4:2:0 format. Eight-by-8 chrominance block 507 represents an up sampling back to the 4:4:4 format from 4-by-4 chrominance block 505. A corresponding edge can not be observed in the up-sampled 8-by-8 chrominance block 507. This quality degradation is visible in non-stationary areas such as edges and fine textures. Details are lost through the down-sampling and up-sampling processes. The receiver wants to see an image as close to the original as possible. Receiver quality is one of the most important criteria when people develop an image/video coding system. Thus the lost information of the edge in the example of FIG. 5 should be restored. This application refers to this problem as the “chrominance mis-alignment” problem. This invention is a proposed technique to re-align these mis-aligned signals. The solution of this invention is described below.